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Abstract. In this paper we shall consider one parametric generalization of some non- 
symmetric divergence measures. The non- symmetric divergence measures are such as: 
KuUback-Leibler relative information, — divergence, relative J - divergence, relative 
Jensen - Shannon divergence and relative Arithmetic ~ Geometric divergence. All the 
generalizations considered can be written as particular cases of Csiszar's f-divergence. 
By putting some conditions on the probability distribution, the aim here is to develop 
bounds on these measures and their parametric generalizations. 



1. Introduction 



Let 



P = {Pl,P2, ■■■,Pv 



Pi > 0,^Pi = 1 > , n^2, 



be the set of all complete finite discrete probability distributions. There exist many 
information and divergence measures in the literature on information theory and statistics. 
Some of them are symmetric with respect to probability distributions, while others are 
not. Here, in this paper, we shall work only with non-symmetric measures. Through out 
the paper it is understood that the probability distributions P, Q G r„. 

l.L Non-Symmetric Measures. Here we shall give some non-symmetric measures of 
information. The well known among them are divergence and KuUback-Leibler rela- 
tive information. 

• x^— Divergence (Pearson [T7^) 

(1) x^(^iiQ) = E^^^^ = Ef -1- 



i=l 
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• Relative Information (KuUback and Leiber jT3]) 

n 

(2) K{P\\Q) = Y,pM{^-). 

i=i 

• Relative J— Divergence (Dragomir et al. [S]) 

(3) D{P\\Q) = ^^{p,-q,) In [P^j. 

• Relative JS— Divergence (Sibson [TH] ) 

(4) — - 



F(P||g) = f^p,ln (- 
i=i 



Pi + Qi 

• Relative AG-Divergence (Taneja ^Hj ) 

(5) G(P||Q)=t(a±*),„(«±* 

The symmetric versions of the above measures are given by 

(6) ^{P\\Q) = x\P\\Q) + x\Q\\P). 

(7) J{P\\Q) = K{P\\Q) + KmP) 

= D{P\\Q) + DmP). 



m\Q) = \[F{P\\Q) + FmP)] 



and 



(9) nP\\Q) = \[G{P\\Q) + G{Q\\P)]. 
After simplification, we can write 

(10) j{p\\Q) = mp\\Q)+np\\Q)]- 

Dragomir et al. [TT] studied the measures We call it [211 by symmetric chi-square 
divergence. The measure © is well known Jeffreys-Kullback-Leiber ^B], ^3] J-divergence. 
The measure (jHI) is information radius studied by Sibson [T^ . It is also known by Jensen 
Shannon divergence measure (ref Burbea and Rao [2]). The measure is new in the 
literature and is studied for the first time by Taneja [23] and is called arithmetic and 
geometric mean divergence measure. Lin ^H] studied some interesting properties and 
applications of the measure El More details on some of these divergence measures can be 
seen in Taneja [23] and in on line book by Taneja [221 • inequality among the 
measures ([El)-® can be seen in Taneja [2j^ 
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In this paper our aim is to work with one parametric generahzation of non-symmetric 
divergence measures given by (jH) and (jSj). A similar kind of study of the measures 
and with their one parametric generahzations can be seen in Kumar and Taneja j^ . 

It is important to note that defining a generalized divergence by introducing a real 
parameter allows to unify many of the known divergence measures studied individually 
and also yields a number of new divergences. The properties and bounds established for 
this family do hold good in particular for member divergences. It provides a vast horizon 
of divergences for users to choose that deems best for their applications. A few examples 
of new divergence measures obtained from this generalized divergence are cited in the 
next section 2. 



2. Generalized Non-Symmetric Divergence Measures 



A one parametric generalization of the measure (eql) can be seen in in Liese and Vajda 
. We refer it here as relative information of type s. 



Relative Information of Type s 



for all s G 



UP\\Q) 



K^iPWQ) = [sis - i)r 

^(Qiin = i:?an(|) 

i=l ^ ^ 



E PiQi 

i=l 



K{P\\Q) = J:P^^^ 
i=l 



Sy^O,l 

s = 

s = 1 



The measure flllj) admits the following particular cases: 

(i) $_i(P||g) = ix^(Q||P). 

(ii) $o(P||Q) = K(Q||P). 

(iii) $i/2(F| \Q) = 4 [1 - B{P\ \Q)] = Ah{P\\Q). 

(iv) ^,{P\\Q) = KiP\\Q). 

(v) UP\\Q) = h'iP\\Q)- 

Thus we observe that MP\\Q) = $-i(Q||P)and $i(P||Q) = MQ\\P)- 
The measures B{P\\Q) and h{P\\Q) appearing in part (iii) are given by 

n 

(12) B{P\\Q) = Y,VP^^ 

1=1 

and 

n 

(13) h{P\\Q) = 1 - B{P\\Q) = (v^- v^)^ 



1=1 
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respectively. 

The measure B{P\\Q) is famous as Bhattacharyya [T] coefficient and the measure 
h{P\\Q)is known as HeUinger ^2] discrimination. 

Now we shall give one parametric generalization of the measures given by Q and (^. 
These generalizations are based on the measure (fTTj) . 

• Unified Relative AG and JS - Divergence of Type s 

Let us consider the following unified one parametric generalizations of the measures (0J 
and © simultaneously. 



(14) n,{p\\Q) 



n 
i=l 



FGs{P\\Q) = [s{s-l)]-' 

i=l ^ 

G(P||Q) = E(^)ln(^ 



2p, 



^7^0,1 

s = 

s = 1 



The measure (I14j) admits the following particular cases: 

(i) Q^,{P\\Q) = \A{P\\Q) 

(ii) no{P\\Q) = F{P\\Q). 

(iii) n,^,{P\\Q)=4[l-B{P\\^)]=Ah{P\\^). 

(iv) n,iP\\Q) = GiP\\Q). 

(v) ^2{p\\Q) = h'iQm- 



The expression A(P||Q) appearing in part (i) is the well known triangular discrimina- 
tion, and is given by 



(15) 



i=l ^ 



The new divergences can be obtained from Qs{P\\Q) by considering different choices 
of the real parameter s. For example, s = and s = — 2 in ([1^1 respectively result the 
following new divergence measures: 



(16) 
and 
(17) 



^^-i/2(P||Q) = 3 



n 

.1=1 



Pi + Qi 



^-2{P\\Q) 



Y^Pi 



1=1 



_2p^ 

Pi + qi 
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In this paper we shall study some properties of the unified generalized measure (|14j). 
Some properties and application of the this measure following the lines of Lin ^H] shall be 
dealt elsewhere. Some applications of this new generalized measure (fT^ towards, pattern 
recognition, statistics, minimization problem, etc. are also under study. 



3. Csiszar's /-Divergence and its Properties 

Given a function/ : [0, oo) M, the f-divergence measure introduced by Csiszar's jlj 
is given by 



;i8) CfiP\\Q) = J2QJ 



i=l 



for all F, Q e r„. 

Here below are some theorems giving properties of the measure (fTHj) . 

Theorem 1. {Csiszar's [HE]) // the function f is convex and normalized, i.e., /(I) = 

0, then the f-divergence, Cf{P\\Q) is nonnegative and convex in the pair of probability 
distribution {P,Q) G r„ x r„. 

Theorem 2. {Dragomir [HI El)- Let f : M_|_ — M 6e difjerentiable convex and normalized 

1. e., /(I) = 0. Then 

(19) Q^Cj{P\\Q)^Ec,{P\\Q) 
where 

n 



(20) Ec,{P\\Q) = J2(P^'<l^)f'^ 



for all P,Q e r„. 

Let P,Q ETn be such that there exists r, R with 0<r^^^R< oo, Vi G {1,2,..., n}, 
then 

(21) O^CfiP\\Q)^Ac,.{r,R), 
where 

(22) Ac,{r,R) = \{R-r)[nR)-f{r)]. 
Further, if we suppose that 0<r^l^R<(yo,ry^R, then 

(23) O^Cf{P\\Q)^Bcf{r,R), 
where 

(24) B,,ir.K^JJ^^Ml}±(l^l}m. 
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Moreover, the following inequalities hold: 
(25) Ec,{P\\Q)^Ac,{r,R), 



(26) 
and 
(27) 



Bc,{r,R)<:Ac,{r,R) 



^ Bc,{r,R)-Cf{P\\Q) ^ Ac,{r,R). 

The inequalities pKjl and (P7j) can be seen in Dragomir while the inequality (jSHI) 
can be proved easily. 

Theorem 3. Let P,Q ETn be such that 0<r^^^R< oo, Vz G {1, 2, n}, for some 
r and R with < r < 1 < R < oo. Let f : ^ M. be differentiable convex, normalized, 
of bounded variation, and second derivative is monotonic with f" absolutely continuous 
on [r, R] and f" G Lao[r, i?], then 



f28) 



and 
(29) 



where 
(30) 

(31) 

(32) 
(33) 



Cf{P\\Q)-]^Ec,{P\\Q) 

^minjlM/) [nR)-nr)]x\P\\Q), 

^ liriL \X\\P\\Q). IfiR) - f'{r)] V{P\\Q) \ , 



Cf{P\\Q)-E*a^XP\\Q) 



^min<j-fc(/) [nR)-nr)]x'{P\\Q), 

^ liriloo IxfiPWQ), \ [f\R) - f{r)] V{P\\Q) !> , 



n . 

E*c^{P\\Q) = J2iP^-Q^)f'( 
i=l ^ 

\xm\Q) = ±^-^ 



, f Pi + Qi 



i=l 



V{P\\Q) = J2\P—<1^1 
1=1 

, = ess sup I /"'I , 

xe[r,R] 



II r 
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and 
(34) 



Kf) 



— 1, if f" is monotonically decreasing 
1, if f" is monotonically increasing 

The above theorem is a combination of different papers due to Dragomir et ah [HlimiTUj. 



The measures ((T)), (jHH) and (j22I) are the particular cases of Vajda "SIT \x\'^— divergence 
given by 



[P\\Q) = Y1 



m ^ 1. 



i=l 



The measure flH^ satisfy the following [3 |H1 properties 



(36) |xr(i^llQ)^^^T#^[(l- 



(R-r) 



m ^ 1. 



and 
(37) 



^ y(p||g) ^ ixr(^IIQ) ^ (^^-^ ] v{P\\Q), m^i. 



(38) 

(39) 
and 
(40) 



For m = 2, m = 3 and m = 1 in we have 

{R-rf 



x\P\\Q)^{R-m-r)^ 



4 



respectively. 

In view of the last inequalities given in (jHHj) . (jHH) and pUj) the bounds given in 
and (j^^ can be written in terms of r, i? as 



(41) 



and 



Cf{P\\Q)--Ec,{P\\Q) 
^ ^^^rmJhif) [f'XR) - fir)] , ^ ||n 



R-r 



(42) CKP||Q)-i?c,(^IIQ) 



< ^^T^ min (1^/) [r(^) - /"(r)] , ^ iin 



48 



f (i?) - fir) 
R-r 



respectively. 
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From now onwards, unless otherwise specified, it is understood that, if there are r, i?, 
then 0<r^^^/2<oo, ViG {1,2, ...,n}, with 0<r<l<R<oo, P = 
iPi,P2, ■■■■,Pn) £ r„ and Q = {qi,q2, ....,qn) G ^n- Also, throughout the paper we shall 
make use of the p-logarithmic power mean [12] given by 



(43) 

for all p G 

(44) 

for all p G 



Lp{a,b) 



fcP+i_nP+i 
(p+l)(fe-a) 
b—a 



In 6— In a ' 

1 

b—a 



p = —1 
p = 



a ^ b. In particular we shall use the following notation 



Ll{a,b) 



(p+l)(fe-a) ■ 
In fe— In a 
b—a 



p^-l 
p= -1 , 

p = 



I, a b. 

4. Relative AG and JS - Divergence of Type s 



In this section we shall consider the measures given by (jl4p and shall give some prop- 
erties. 



Let us consider 



(45) 



s = 
s = 1 



¥+(^)ln(^), 

for all a; > in then Cf{P\\Q) = (F||Q), where (P||Q) is as given by (HH). 
Moreover, 



(46) f,ix) 



and 



s - 1] 

l-x 



2(l+x) 

i[l-x- + ln(^)]. 



s ^ 0, 1 
s = 
s = 1 



4:X' 



1 (x+l\ 
x^ \ 2x J 



s-2 



(47) 



^P':ix) 



1 



x{x+l)^ ' 

1 

2x2 (x+l) , 



s = 

s = 1 



Thus we have ipg{x) > for all x > 0, and hence, '0s(x) is convex for all a; > 0. Also, 
we have ■0s (1) = 0. In view of this we can say that relative AG and JS - divergence of 
type s is nonnegative and convex in the pair of probability distributions (P, Q) G r„ x r„. 

Based on Theorem |2l we have the following theorem. 
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Theorem 4. The following bounds on Qs{P\\Q) hold: 

(48) ns{P\\Q) ^ i?n.(P||Q)(P||Q) ^ An,^P\\Qir,R) 
and 

(49) nsiPWQ) ^ Bn,(^PiiQ){r,R) ^ An4PiiQ)ir, R), 
where 

(50) En4P\\Q)iP\\Q) 
[sis - t im) {V^) ^ + (1 - s)q.] , . ^ 0, 1 



i=l 
2 



D(Q||P)-1A(P||Q), 
1 

k 2 



\[x\p\\Q)-Dmp)]. 



s = 
s = 1 



(51) 

and 
(52) 



An4P\\Q){r, R) 



ArR 



TS-i 



I (r + l R+l 



R 



TS-2 



r+l R+l 
r ' R 



BnsiP\\Q){r, R) 



1 TS-l ( r+l R±l\ 
2(s-l) s-1 I 2r ' 2R ) 

^[rlu(=^)-fil„(i||t)]_ii:;(:±l,2||l). 



rR-1 r-1 ( r+l R+l\ , 1 ]^ f gi+l)(r+iy\ 
, 4rR -1 ^ 2r ' 2_R 2 \^ 4rR J ' 



5^0,1 
s = 

s = 1 



Corollary 1. The following inequality holds: 



(53) 



-A(P||Q)^D(g||P)^X^(Q||P). 



Proof. It follows from PHj) . where we take s = and s = 1 in ()50p. □ 

Theorem |3 admits some particular cases. These are summarized in the following two 
corollaries. 

Corollary 2. The following bounds hold: 



(54) FiP\\Q)^DiQ\\P)-^AiP\\Q) 



(R-r) 
ArR 



I fr+1 R+l 



R 



^-2 



r + l R + l 
r ' R 



10 



PRANESH KUMAR AND INDER JEET TANEJA 



and 
(55) 



G{P\\Q) ^\[x\Q\\P) - Dmp)] 



(R-r) 
8rR 



l-L-\ 



r + 1 R+l 
r ' R 



Proof. In inequalities (jlHI), take s = and s = 1 we get (jH^ and respectively. 

For s = — 1 and s = 2 the results are obvious. 
Corollary 3. The following hounds hold: 

(56) 



{R+l){r + l) ^ {R+lf{r + iy 



(57) F{P\\Q)^ 



R-r 
ArR 



r In 



r + l 
2r 



i?ln 



R+l 
2R 



r + l R + l 
r ' i? 



_i , r + 1 i? + 1 



^-2 



r + l i?+ 1 
r ' R 



□ 



(5^ 



(59) 



G{P\ \Q) ^ -^L-i I J + 2 i^^^R 



{R-r) 
8rR 



1-LZi 



_i fr + 1 R+l 



R 



{R-l){l-r) {R-rf{R + r) 



rR 



4^2 ^2 



Proof In inequalities (j^ . take s = — l,s = 0, s = l and s = 2 we get respectively (j^ . 
dSZI), dSHl) and dSni). □ 

Based on Theorem El we have the following result. 

Theorem 5. The following bounds hold: 



(60) 



QsiPWQ) - -EnAPWQ) 



^mm!^^SnAr,R)x\P\\Q), ^ Ik 
(R-ry , R-r 



\x\'{P\\Q), [f{R)-f'ir)] 



lloo' 



R-r 



[f{R)-f'{r)]V{P\\Q) 
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(61) \ns{P\\Q) - Et,XP\\Q)\ 



1 



where 
(62) 



' 24 



^ min ^ -J^Xr. R)x\P\ IQ), t^j IICIL IxI' {P\\Q). \ IfiR) - fir)] V{P\\Q) 



R-r 



6 



IfiR) - fir)] , 



EnSPWQ) 



/ 1 " 



n / \ " 



Pi+3(5ri \ / Pi + (3-2s)qi 



1=1 ^ 

n ^ ^ 

|A(P||g) + iEfe-g.)in (gfl)), 



1 (Pi-Qif 
1=1 



s 7^0,1 
s = 
s = 1 



(63) 

and 
(64) 



5ns (r, i?) 



1 /r + n" ^ 1 f R+l^ ' ^ 



2r 



V 2R 



s>-l 



I loo 



[s + 1 + 3r) /r + 1 



, s ^ -1. 



r2(r + 1)^ \ 2r 
Proof. The third order derivative of the function ^'^(x) is given by 

s + 1 + 3x) + 1 



(65) 

This gives 
(66) 



x2(x + 1)^ \ 2x 



ijl'ix) ^ 0, V s ^ -1. 



X G (0, oo) 



From (j66p . we can say that the function il)"{x) is monotonically decreasing function 
X G (0, oo), and hence, for all x G [r, i?], we have 



(67) 



5nXr,R)=rir)~riR) 



I (r + lV 1 f R+l^ 



^3 \ 2r 



i?3 V 2R 



s > -1. 



From ()65|1 . we have 
(68) l^f I ' = 



> + l)(s + 2) + 8(s + l)x + 12x2\ /x + 1 



x3(x + l)"^ 



22; 
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From (|68|) . we can say that the function is monotonically decreasing function in 
X G (0, oo) for s ^ —1, and hence, for all x G [r, i?], we have 



(69) 



II oo 



sup ICI 



(s + 1 + 3r) [r + l 



r2(r + l)3 V 2r 



s ^ -1 



Applying Theorem El for the measure ()14|) along with ()67p and (jU^ we get the required 
proof. □ 

In view of the inequalities ()6U|) and we have some particular cases given in the 
following corollary. 

Corollary 4. The following hounds hold: 



(70) 



A(P||Q)-2^g. 
^ min < 2 



i=l 
1 



Pi - g» 
Pi + % 

1 



r + l)3 {R+lf 



x\P\\Q) 



(r + l)4 
^ min { 2{R - r)^ 



(r + l)2(/2 + l)2 
1 



_(r + l)3 (i? + l)3_ 

{R-rf 4:{R-r)^{R + r + 2) 
2(r + l)4' (r + l)2(i?+l)2 



(71) 



n^llQ)-^^(QII^) + ^A(^IIQ) 



< min 



1 



1 



r(r + 1)2 i?(i?+l)2 



/(i'llQ) 



^ min 



R-r 

rR 
{R-rf 

32 
{R-rf 
2rR 



1 , r + 1 R+l 
^-1 



R 



^-2 



3r + 1 



1 



r(r + 1)2 i?(i?+ 1)2 
, /r + 1 



, r + 1 i?+ 1 



R 



12r2(r + 1)3 
r + l i?+ 1 
r ' R 
{3r + l){R-rY 
' 96r2(r + l)3 ' 
2/r + l i? + l 



xl='(P||Q), 
V{P\\Q) 
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and 



(72) 



G{P\\Q) -\[x\Q\\P) - Dmp)] 



< min 



16 



1 



r2(r + l) i?2(i?+l) 



x\P\\Q) 



3r + 2 



< min 



{R-r) 

2rR 
{R-rf 

64 
{R~rf 

ArR 



l-L-_\ 
1 



r + 1 R+l 



r ' 



r2(r + l) R^{R + l) 
l-L-_\ 



24r3(r + 1)2 

V{P\\Q)Y 

(3r + 2)(/?-r)2 
192r3(r + l)2 ' 



Ixl'(i^llQ), 



_i , r + 1 i? + 1 



Proof. Letting in (jUUI) . s = — 1, s = and s = 1 we get the inequalities (fTOI) . (f7T|) and 
(17^ respectively. □ 



In view of the inequalities ()6H) we have some particular cases given in the following 
corollary. 



Corollary 5. The following hounds hold: 



(73) 



A(P||Q)-^(p, + 7g.) 



C min 



1 



Pi - Qi 
Pi + Mi 



< min 



r+l)3 {R+lf 

2{R-r){R + r + 2) 
(r + l)2(i?+l)2 

(i?-r)^ 



(r + l)^ 



Ixl'(i^llQ), 



ViP\\Q)\. 



(r + l)3 (i?+l)= 



(fi-r) 
8(r+ 1) 



4' 



(/2-r)2(i? + r + 2) 
(r + l)2(i?+l)2 
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(74) 



and 
(75) 



F{P\ \Q) -Yiq.- P.) In { + i y 



^ min < - 



r(r + 1] 



< min 



R-r 
2rR 
{R-rf 

32 
{R-rf 
ArR 



R{R+1) 
r + 1 R+1 



x\P\\Q) 



3r + 1 



R 



1 



_r(r + 1)2 i?(i?+l)2 
r + 1 R+1 



R 



^-2 



24r2(r + 1)3 
r + 1 R + 1 
r ' R 
(3r + l)(i?-r)3 
192r2(r + 1)3 
r + 1 i?+ 1 
r ' R 



xf{P\\Q), 
V{P\\Q) 



1 1 " / 

G'(P||Q)--A(P||Q)--5^(p,-g,)ln 



- Qi 
Pi + Mi 



< min 



< min 



1 


1 


16 


r'^{r - 


-1) 


{R 


— r) 


1 - 


ArR 


{R 


-r)2 




64 


r2( 


{R 


-r)2 


1 - 


8rR 



x'{P\\Q), .o^"^^... \xf{P\\Q), 



1 



r + 1 R+1 



R 



r2(r + l) R^{R + 1) 



r+1 R+1 



48r3(r + 1)2 

V{P\\Q)^ 

(3r + 2)(i?-r)2 
384r3(r + l)2 ' 



R 



Proof. Letting in (jF)T|) . s 
(ITHjl respectively. 



-1, s = and s = 1 we get the inequalities (|7H|) . (fTIj) and 

□ 
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